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MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  FOR  ACQUISITION 

AND  TECHNOLOGY 


SUBJECT:  Automatic  Test  Systems  Acquisition  (Project  No.  4AE-5029) 


Introduction 

We  are  providing  this  final  memorandum  report  for  your  information  and  use. 
This  report  discusses  the  efforts  to  achieve  commonality  in  standards  among  the 
Military  Departments  as  part  of  the  DoD  policy  for  automatic  test  systems 
(ATS).  Those  systems  are  comprised  of  automated  test  hardware,  operating 
system  software,  and  test  program  sets.  ATS  are  used  in  DoD  field  and  depot 
electronics  maintenance  activities  to  test  electronic  equipment,  to  reduce 
troubleshooting  times,  and  to  augment  the  skills  of  field  technicians.  When  this 
audit  was  announced  March  28,  1994,  the  Under  Secretary  of  Defense  for 
Acquisition  and  Technology  (USD[A&T])  had  not  established  DoD  policy  and 
procedures  for  ATS  acquisition.  (See  Enclosure  1  for  definitions  of  terms  in 
this  report.) 


Audit  Results 

On  April  29,  1994,  USD(A&T)  issued  a  DoD  policy  memorandum  to  improve 
the  ATS  acquisition  process  (Enclosure  2).  Also,  the  Office  of  the  USD(A&T) 
has  prepared  a  draft  directive  for  the  DoD  ATS  Acquisition  Program  and 
drafted  ATS-related  changes  to  be  in  DoD  Instruction  5000.2,  "Defense 
Acquisition  Management  Policies  and  Procedures,"  February  23,  1991.  These 
policies  and  procedures  for  ATS  acquisition  will  minimize  the  introduction  of 
unique  types  of  ATS  and  encourage  the  use  of  commercial-off-the-shelf  testers 
and  components  throughout  DoD.  According  to  the  Navy,  the  use  of 
commercial-off-the-shelf  testers  and  components  will  reduce  logistics  burdens 
and  long-term  cost  to  DoD.  The  Navy,  as  the  designated  DoD  ATS  Executive 
Agent,  is  also  leading  a  tri-Service  survey  of  all  current  and  ongoing  Milit^ 
Department  ATS  programs  for  application  of  the  ATS  policy  and  is  evaluating 
requirements  for  ATS  research  and  development  funding. 


Objectives 

The  overall  audit  objective  was  to  evaluate  the  DoD  efforts  to  achieve 
commonality  in  standards  among  the  Military  Departments  as  part  of  DoD 
automatic  test  systems  investment  strategy.  The  audit  also  assessed  the 
adequacy  of  applicable  internal  controls. 


Scope  and  Methodology 

We  conducted  this  program  audit  from  March  1994  through  September  1994 
and  reviewed  data  dated  from  January  1989  through  August  1994.  To 
accomplish  the  objectives,  we  interviewed  and  obtained  program  documentation 
from  the  Office  of  the  Secretary  of  Defense  (OSD)  and  Military  Department 
officials  involved  with  ATS.  Specifically,  we  discussed  and 

o  reviewed  DoD  ATS  policy  and  procedures  for  monitoring  the 
acquisition  and  management  of  maintenance  and  diagnostic  test  equipment, 

o  evaluated  ATS  research  and  development  funding  and  related 
documents, 

o  reviewed  the  formulation  of  ATS  directives  resulting  from  DoD 
policy  for  ATS,  and 

o  reviewed  the  DoD  Automatic  Test  Systems  Investment  Strategy 

Study. 

The  audit  was  made  in  accordance  with  auditing  standards  issued  by  the 
Comptroller  General  of  the  United  States,  as  implemented  by  the  Inspector 
General,  DoD,  and  accordingly  included  such  tests  of  internal  controls  as  were 
deemed  necessary.  We  did  not  rely  on  computer-processed  data  to  support  our 
audit  results.  Enclosure  4  lists  the  organizations  visited  or  contacted. 


Internal  Controls 

Internal  Controls  Evaluated.  We  evaluated  internal  controls  related  to  the 
effectiveness  of  the  ATS  acquisition  process  and  the  adequacy  of  ATS  to 
achieve  commonality  among  the  Military  Departments  as  part  of  the  DoD  ATS 
Investment  Strategy.  We  also  assessed  the  implementation  of  the  requirements 
in  DoD  Directive  5010.38,  "Internal  Management  Control  Program,"  April  14, 
1987,  including  performance  of  vulnerability  assessments  and  management 
control  reviews. 

Internal  Control  Weakness  Not  Identified.  We  did  not  identify  any  material 
internal  weakness,  as  defined  by  DoD  Directive  5010.38;  however,  the  Military 
Departments  had  not  yet  performed  vulnerability  assessments  and  management 
control  reviews  on  ATS.  Personnel  from  OSD  and  the  Military  Departments 
indicated  that,  since  they  were  still  formulating  DoD  ATS  policies  and 
procedures,  performing  the  assessments  and  reviews  before  establishing  policies 
and  procedures  would  be  premature. 


Prior  Audits  and  Other  Reviews 

Since  1989,  the  Inspector  General,  DoD,  has  issued  four  reports  that  address 
ATS.  Enclosure  3  discusses  those  four  reports  and  the  status  of  management's 
implementation  of  report  recommendations. 


2 


Background 

Automatic  Test  Systems.  Automatic  test  systems  (ATS)  are  used  to  test 
weapon  system  electronics  during  manufacturing  and  maintenance.  DoD  ATS 
are  technically  complex  and,  in  some  cases,  as  costly  as  the  weapon  systems 
they  support,  from  $1  billion  to  $2  billion.  Consequently,  Congress  has 
address^  the  need  to  establish  ATS  commonality  in  acquisition  of  ATS  among 
the  Military  Departments. 

Congressional  Direction  for  FY 1993.  The  House  of  Representatives 
Conference  Report  No.  102-1015,  "Making  Appropriations  for  the  Department 
of  Defense  for  the  Fiscal  Year  Ending  September  30,  1993,  and  for  Other 
Purposes,"  Octobers,  1992,  addressed  automatic  test  equipment  and  directed 
that  OSD: 

o  consider  the  current  families  of  Air  Force  support  equipment,  the 
Navy  Consolidated  Automated  Support  System  (CASS),  and  the  Army 
Integrated  Family  of  Test  Equipment  (IFTE)  when  establishing  a  common 
family  of  test  equipment; 

o  develop  a  DoD-wide  policy  r^uiring  automatic  test  equipment 
commonality  in  standards  among  the  Military  Departments  along  with  an 
oversight  system  to  ensure  compliance; 

o  ensure  that  the  Air  Force  establishes  a  Secretarial  mandate  and  control 
procedures  for  standardized  test  equipment;  and 

o  provide  a  report  by  April  1,  1993,  that  addresses  DoD's  ATS 
investment  strategy  study,  the  strategy's  potentijil  savings,  and  how  the  strategy 
is  being  implemented. 

DoD's  Automatic  Test  Systems  Investment  Strategy  Study.  On  May  20, 
1993,  the  Assistant  Secretary  of  Defense  for  Production  and  Logistics^  (the 
Assistant  Secretary)  provided  a  partial  response  to  the  conference  report.  TTiis 
response  provided  the  results  of  DoD's  ATS  investment  strategy  study,  which 
was  conducted  by  a  team  consisting  of  personnel  from  OSD,  the  Military 
Departments,  and  the  Institute  for  Defense  Analysis.  The  study  indicated  that 
DoD  had  invested  an  estimated  $35  billion  in  ATS  from  1980  through  1992, 
with  an  additional  $15  billion  estimated  for  associated  ATS  support.  The  DoD 
plans  to  spend  another  $11  billion  for  ATS  through  1999  if  acquisition  strategy 
changes  are  not  adopted.  If  DoD  factory  test  equipment  investments  are 
included,  the  ATS  expenditure  may  total  more  than  $100  billion  from  1980 
through  1999.  The  study  concluded  that  implementing  an  investment  strategy 
that  uses  designated  DoD-standard  ATS  families,  instead  of  weapon  system- 
unique  ATS,  should  reduce  ATS  acquisition  cost.  The  cost  reduction  would 
range  from  15  to  20  percent  from  FYs  1993  through  1999,  resulting  in  a 
potential  net  saving  of  more  than  $1.5  billion.  The  study  further  concluded  that 

Ijn  September  1993,  OSD  disestablished  this  office  and  created  the  Offices  of  the  Assistant 
Secretary  of  Defense  (Economic  Security),  the  Deputy  Under  Secretary  of  Defense 
(Environmental  Security),  and  the  Deputy  Under  Secretary  of  Defense  (Logistics). 
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dispersed  DoD  ATS  investment  decisions  within  and  across  Military  Department 
and  weapon  program  lines  is  a  major  contributing  factor  to  DoD  ATS 
proliferation. 

The  Assistant  Secretary  also  indicated  that  needed  policy  revisions  were  being 
developed  and  that  the  Military  Departments  were  coordinating  ATS  strategy 
implementation.  He  planned  to  conclude  the  response  to  the  conference  rejwrt 
during  the  fourth  quarter  of  FY  1993.  Further,  he  stated  that,  in  the  interim. 
Acquisition  Category  I  and  other  Defense  Acquisition  Board  level  interest 
programs  would  be  assessed  for  ATS  strategy  application  as  a  part  of  Defense 
Acquisition  Board  reviews. 

According  to  an  official  from  the  Office  of  the  Deputy  Under  Secretary  of 
Defense  (Economic  Security),  the  USD(A&T)  provided,  in  June  1994,  a 
follow-up  response  to  the  conference  report.  The  official  stated  that  Senate  and 
House  Appropriation  Committee  staffers  advised  him  that  Congress  did  not 
require  OSD  to  provide  further  ATS-related  responses.  We  contacted  a  Senate 
Appropriations  Committee  staffer  to  determine  whether  Congress  required 
OSD,  as  of  September  1994,  to  provide  further  responses.  The  staffer  indicated 
that  additional  OSD  responses  were  not  required  since  OSD  issued,  in  Janu^ 
1994,  the  DoD  ATS  investment  strategy  study  and  was  developing  DoD-wide 
ATS  policy. 

Congressional  Direction  for  FY  1994.  The  House  of  Representatives 
Committee  on  Appropriations  Report  No.  103-254,  "Department  of  Defense 
Appropriations  Bill,  1994,"  September  22,  1993,  provided  additional 

congressional  direction  on  DoD's  ATS  program.  The  Committee  recommended 
that  the  Under  Secretary  of  Defense  for  Acquisition:^ 

o  review  and  approve  immediately  DoD's  proposed  acquisition  policy 
that  defines  and  requires  the  use  of  ATS  for  weapon  system  and  equipment  test 
and  maintenance, 

o  establish  a  separate  research  and  development  program  for  future  ATS 
technology  for  needed  family  modifications  and  research  and  development  for 
next  generation  ATS,  and 

o  submit  an  OSD-approved  cost-benefits  analysis  to  the  congressional 
defense  committees  before  developing  and  procuring  weapon  system-unique 
ATS. 


Discussion 

DoD  Policy  Formulation.  In  FY  1994,  the  Office  of  the  USD(A&T)  began 
formulating  DoD  policy  and  procedures  for  ATS  acquisition  in  response  to 
congressional  directions  and  previous  audit  reports  from  the  Office  of  the 
Inspector  General,  DoD.  In  an  April  29,  1994,  policy  memorandum,  the 


^Renamed  USD(A&T)  in  November  1993. 
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USD(A&T)  appointed  the  Navy  as  the  DoD  Executive  Agent  for  ATS  and 
directed  that  it  provide  the  Director,  Weapon  Support  Improvement  Group, 
Office  of  the  Assistant  Secretary  of  Defense  (Economic  Security),  Office  of  the 
USD(A&T): 

o  a  coordinated  Executive  Agent  Charter  to  issue  as  a  DoD  directive; 

o  proposed  acquisition  changes  to  incorporate  in  DoD  Instruction 
5000.2; 

o  recommended  organizational  and  funding  adjustments  to  implement 
ATS  policy;  and 

o  a  survey,  which  was  to  be  completed  by  October  29,  1994,  of  all 
current  and  ongoing  Military  Department  ATS  programs  for  application  of  the 
new  policy. 

On  August  29,  1994,  the  DoD  ATS  Executive  Agent  Office  forwarded  the 
propos^  draft  directive  and  DoD  Instruction  5000.2  language  changes, 
including  recommended  organizational  and  funding  adjustments,  to  the  Assistant 
Secretary  of  the  Navy  (Research,  Development  and  Acquisition)  for 
endorsement  to  USD(A&T).  When  we  concluded  this  audit  in  September  1994, 
the  Office  of  the  USD(A&T)  had  granted  the  Navy  a  30-day  extension  to 
complete  the  draft  directive  and  language  changes.  However,  on  October  31, 
1994,  when  we  finalized  this  audit,  the  Navy  had  not  yet  forwarded  the  draft 
directive  <md  language  changes  to  USD(A&T).  The  Navy  was  also  coordinating 
a  request  to  USD(A&T)  for  a  30-day  extension  to  complete  the  survey  of  all 
current  and  ongoing  Military  Department  ATS  programs. 

Proactive  Policy  Implementation.  To  implement  the  ATS  policy 
memorandum,  OSD  and  the  Military  Departments  established  working  groups 
and  were  coordinating  ATS  policy. 

Working  Group  Establishment.  The  DoD  ATS  Executive  Agent 
Organization  established  working  groups  to  assist  the  DoD  ATS  management 
board,  which  includes  representatives  from  the  ATS  Executive  Agent  Office  and 
each  Milit^  Department.  The  ATS  policy  working  groups  are  accomplishing 
the  tasks  directed  in  the  April  29,  1994,  memorandum  and  developing  a  waiver 
review  process. 

Policy  Coordination.  Officials  in  the  Office  of  the  Director,  Weapon 
Support  Improvement  Group,  indicated  that  they  plan  to  coordinate  the  draft 
ATS  policy  and  procedures  with  the  same  principal  parties  with  which  the 
April  29,  1994,  memorandum  was  coordinated.  Because  the  memorandum  was 
not  coordinated  with  the  Office  of  the  Director,  Operational  Test  and 
Evaluation,  we  suggested,  and  Weapon  Support  Improvement  Group  officials 
agreed,  that  the  Office  of  the  Director,  Operational  Test  and  Evaluation,  should 
be  included  in  the  coordination  of  the  ATS  acquisition  policy  and  procedures. 

Adequacy  of  ATS  to  Achieve  Commonality.  The  DoD  ATS  investment 
strategy  study  included  an  in-depth  review  of  data  on  ATS  supporting  15  DoD 
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weapon  systems.  Analyses  were  made  to  assess  technical  capabilities  of  modem 
DoD  ATS  and  to  determine  whether  DoD  can  meet  cross-weapon  system  and 
cross-Service  test  r^uirements,  using  standard  ATS  families.  The  study 
identified  that  technical  testing  capabilities  of  unique  ATS  had  considerable 
overlap.  In  the  past,  duplicate  ATS  capabilities  often  resulted  because  DoD 
individual  program  and  depot  managers  made  ATS  investment  decisions  in  a 
decentralized  and  uncoordinated  manner.  Based  on  the  analyses,  the  study 
concluded  that  existing  general  purpose  ATS,  comprised  of  IFTE  and  CASS, 
can  meet  an  estimated  95  percent  of  DoD's  technical  test  requirements  for  the 
next  decade,  with  minimal  modifications. 


Conclusion 

The  DoD  ATS  investment  decisions  will  become  more  focused,  resulting  in 
reductions  of  duplicated  ATS  capabilities,  when  the  DoD  ATS  investment 
strategy  changes  are  adopted  and  the  DoD  ATS  acquisition  policy  and 
procedures  that  are  being  developed  are  completed.  Specifically,  the 
implementation  of  a  new  DoD  ATS  Investment  Strategy  and  ATS  policy  and 
procedures  will: 

o  minimize  unique  types  of  ATS  being  introduced  into  DoD  field, 
depot,  and  manufacturing  operations  by  focusing  ATS  development  and  upgrade 
investments  on  a  small  number  of  ATS  families; 

o  increase  the  use  of  commercial-off-the-shelf  testers  and  components 
by  eliminating  duplicate  Defense-unique  ATS  industrial  capabilities  and  using 
widely  used  commercial  and  dual-use  test  capabilities;  zmd 

o  reduce  logistics  burdens  and  long-term  costs  to  DoD. 

The  considerable  cost  for  research  and  development  and  procurement  of  unique 
types  of  ATS  called  for  a  change  in  the  current  ATS  acquisition  process.  As  we 
discussed,  DoD  has  taken  actions  to  establish  a  cost-effective  ATS  acquisition 
program  by  formulating  ATS  policy  and  determining  the  adequacy  of  designated 
standard  ATS  families  to  meet  DoD  automatic  testing  technic^  requirements. 


Management  Comments 

We  provided  a  draft  of  this  report  to  the  addressee  on  September  30,  1994. 
Because  the  report  made  no  recommendations,  official  comments  were  not 
required.  However,  on  October  27,  1994,  we  received  informal  comments 
from  the  DoD  ATS  Executive  Agent  Office  (the  Navy).  We  considered  those 
comments  in  preparing  this  final  report.  The  comments  suggested  minor  factual 
changes  to  the  draft  report.  Also,  the  Navy  advised  that  it  had  not  yet 
completed  the  Executive  Agent  Charter  for  issuance  as  a  DoD  directive,  the 
DoD  Instruction  5000.2  language  changes,  and  the  survey  of  all  current  and 
ongoing  Military  Department  ATS  programs.  When  OSD  and  the  Navy 
complete  those  documents  and  the  survey,  we  request  that  the  Navy  provide  us 
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with  copies  of  the  DoD  directive  and  the  DoD  Instruction  5000.2  language 
changes  and  an  executive  summary  of  the  survey  results.  This  report  does  not 
have  a  quantifiable  financial  impact. 


We  appreciate  the  courtesies  extended  to  the  audit  staff.  If  you  have  any 
questions  on  this  report,  please  contact  Mr.  John  E.  Meling,  Program  Director, 
at  (703)  604-9091  (DSN  664-9091)  or  Mr.  Jack  D.  Snider,  Project  Manager,  at 
(703)  604-9087  (DSN  664-9087).  We  list  the  audit  team  members  inside  the 
back  cover.  Enclosure  5  lists  the  distribution  of  this  report. 

Robert  J.  Lieberman 
Assistant  Inspector  General 
for  Auditing 

Enclosures 
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Definitions  of  Terms 


Acquisition  Category.  A  classification  established  to  facilitate  decentralized 
decisionmaking  and  execution  and  compliance  with  statutorily  imposed 
requirements.  The  categories  determine  the  level  of  review,  decision  authority, 
and  applicable  procedures  and  range  from  Acquisition  Categories  I  to  IV. 

Automatic  Test  Equipment.  Automatic  test  equipment  (ATE)  include  an 
operating  system  or  software  and  a  range  of  hardware  components.  The 
components  consist  of  items  such  as  a  main  computer  within  the  test  equipment, 
an  operating  system,  stimulus  and  measurement  instruments,  signal  control  and 
switches,  and  interfaces  with  heating  and  cooling  sources  and  structural  support. 

Automatic  Test  Systems.  Automatic  test  systems  (ATS)  are  comprised  of  ATE 
hardware  and  operating  system  software  and  test  program  sets  that  include 
hardware  connectors  and  software  programs  to  test  individual  weapon  electronic 
items.  The  ATS  are  used  in  DoD  field  and  depot  electronics  maintenance 
activities  to  test  electronics  technologies  that  are  difficult  or  impossible  to  test 
manually,  to  reduce  troubleshooting  times,  and  to  augment  the  skills  of  field 
technicians.  ATS  are  also  used  in  manufacturing  and  acceptance  testing. 

Consolidated  Automated  Support  System.  The  Navy's  Consolidated 
Automated  Support  System  (CASS)  Program  is  composed  of  standardized  ATE. 
The  ATE  is  structured  around  a  common  core  with  four  testing  configurations 
with  computer-assisted,  multi-function  capabilities  to  support  testing  of  aircraft 
subsystems  and  missiles.  The  CASS  Program  features  fleet-wide 
standardization  of  hardware  and  software  elements  and  is  designed  to  enhance 
electronic  test  capability  over  existing  ATE.  The  CASS  Program's  system 
hardware  is  also  designed  to  provide  the  Navy  intermediate  and  depot-level 
maintenance  with  better  electronic  testing  capability.  The  CASS  Program  is 
composed  of  functional  interfaces  allowing  the  testing  of  a  variety  of  hardware. 
The  interfaces  are  evolving  to  meet  Air  Force  as  well  as  Navy  requirements. 
To  minimize  unique  types  of  automatic  test  systems,  the  Under  Secretary  of 
Defense  for  Acquisition  and  Technology  has  established  policy  to  require  DoD 
Components  to  satisfy  all  acquisition  needs  for  ATE  hardware  and  software  by 
using  designated  automatic  test  systems.  The  CASS  Program  is  one  of  those 
designated  systems. 

Defense  Acquisition  Board.  The  Defense  Acquisition  Board  is  the  senior 
advisory  body  to  the  Under  Secretary  of  Defense  for  Acquisition  and 
Technology  to  advise  the  Under  Secretary  in  enforcing  policies  and  procedures 
governing  the  operations  of  the  DoD  Acquisition  System.  The  Defense 
Acquisition  Board  is  the  primary  forum  to  advise  the  Under  Secretary  on 
mission  needs  approved  by  the  Joint  Requirements  Oversight  Council,  possible 
Concept  Exploration  or  Definition  study  efforts,  and  milestone  decision  point 
reviews  and  program  reviews  of  major  Defense  acquisition  programs  subject  to 
Defense  Acquisition  Board  review.  The  reviews  ensure  that  a  program  is  ready 
to  proceed  into  more  advanced  stages  of  development  or  production  before 
receiving  milestone  approval  and  that  proposed  program  plans  for  subsequent 
stages  are  consistent  with  sound  acquisition  management  practices. 
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Definitions  of  Terms 


Department  of  Defense  Acquisition  System.  A  single  uniform  system  by 
which  all  equipment,  facilities,  and  services  are  planned,  designed,  developed, 
acquired,  maintained,  and  disposed  of  within  the  DoD.  The  system 
encompasses  establishing  and  enforcing  policies  and  practices  that  govern 
acquisitions,  to  include  documenting  mission  needs  and  establishing 
performance  goals  and  baselines,  determining  and  prioritizing  resource 
requirements  for  acquisition  programs,  planning  and  executing  acquisition 
programs,  directing  and  controlling  the  acquisition  review  process,  developing 
and  assessing  logistics  implications,  contracting,  monitoring  the  execution  status 
of  approved  programs,  and  reporting  to  Congress. 

Department  of  Defense  Standard  Automatic  Test  System  Family.  An 
automatic  test  system  with  general  purpose  capabilities  that  meet  the  testing 
needs  of  multiple  DoD  systems  and  has  been  designated  by  one  or  more  of  the 
Military  Departments  as  a  common  automatic  test  system  for  multiple  weapon 
system  testing  applications.  DoD  has  identified  the  Army  Integrated  Family  of 
Test  Equipment  Base  Shop  Test  Facility  and  the  Navy  Consolidated  Automatic 
Support  System  as  the  two  existing  ATS  standard  families. 

Integrated  Family  of  Test  Equipment.  The  Army  initiated  the  Integrated 
Family  of  Test  Equipment  (IFTE)  to  reduce  the  proliferation  of  unique  ATE 
that  it  was  procuring  for  weapon  systems.  Under  IFTE,  the  Army  designed 
standard  ATE  to  provide  multi-functional  testing  capabilities  of  electronic 
components  for  major  weapons  systems.  The  IFTE  is  composed  of 
five  subsystems  of  ATE.  The  primary  intermediate  maintenance  level 
subsystem  is  the  Base  Shop  Test  Facility  (the  Test  Facility)  that  is  installed  in  a 
shelter  and  mounted  on  a  5-ton  truck  for  Army  use.  The  Test  Facility  is  general 
purpose,  multi-functional  ATE  that  uses  several  configurations  of  tests  to 
diagnose  electronic  faults  or  failures  in  weapon  systems.  To  minimize  unique 
types  of  automatic  test  systems,  the  Under  Secretary  of  Defense  for  Acquisition 
and  Technology  has  established  policy  to  require  DoD  Components  to  satisfy  all 
acquisition  needs  for  ATE  hardware  and  software  by  using  designated  automatic 
test  systems.  The  IFTE  is  one  of  those  designated  systems. 

Test  Program  Sets.  Interface  test  adapters,  test  program  software,  and  test 
program  documentation  and  data.  The  test  program  set  provides  necessary 
resources  and  information  to  test  an  item  on  ATE. 
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ACQUWnOM  AND 
TmCMNOCJOOV 


THE  UNDER  SECRETARY  OP  DEFENSE 
3010  DEFENSE  FENTAOON 
WASHINOTON.  O.C.  303013010 


t  •  UPR  1994 


MEMDRANDOK  FOR  ASSISTAOT  SSCREIARY  OF  TOE  ARMVT  (R£5EARC», 

oevELOFMorr,  and  acquisition) 

ASSISTANT  SECRETARY  OF  THE  NAVY  (RESEARCH. 
CEVELOFHENT  AND  ACQUISITIGN} 

ASSISTANT  SECRETARY  OF  TOE  AIR  FORCE  (AOQCaSITlON) 


SUBJECT;  DoD  Policy  for  Autoroatic  T33t  Sy8t«ns 


I  am  ostablishing  a  policy  that  DoD  conponenta  shall  satisfy  all 
acquisition  nssds  for  automatic  tact  equipment  hardware  and  software 
by  uaing  designated  automatic  test  system  (ATS)  families.  This  policy 
is  being  undertaken  to  minimize  unique  types  -of  ATS  being  introduced 
into  DoD  field,  depot,  and  manufacturing  operations,  and  to  encourage 
lAse  of  commercial-off-the-shelf  (COTS)  testers  end  cooponents, 
thereby  reducing  logistics  burdens  and  long-term  costs  to  the  DoD. 

ATS  capabilities  shall  be  defined  through  control  of  critical 
hardware  and  software  elements  and  interfaces  to  ensure  DoD  family 
tester  and  COTS  tester  and  cocrponcnt  interoperability,  and  to  meet 
future  DoD  test  needs.  Based  on  the  recoroendations  of  a  joint 
Service  ATS  Investment  strategy  group,  the  Army's  Integrated  Family 
of  Test  Equipment  (IFTE)  and  the  Navy's  Consolidated  Automated 
Support  System  (CASS)  are  designated  as  the  initial  DoD  families. 

For  weapon  programs  which  require  no  field  AT$,  depot  and  factory 
sell-off  ATS  needs  may  be  met  by  use  of  COTS  testers  and  components, 
along  with  the  defined  critical  elements  end  interfaces. 

ATS  will  now  be  managed  as  a  separate  standardized  ccmmodity. 

The  Navy  is  appointed  as  the  DoD  Executive  Agent  for  ATS.  The 
ficecutlve  Agent's  responsibilities  will  include:  def lining  end 
managing  DoD  ATS  standards.  ATS  family  product  engineering  end 
requirsments.  ATS  specification  end  procursment  review,  imiver 
process  for  OUSD(A&T).  end  Lead  Standardization  Activity. 

The  Navy  is  requested  to  establish  an  ATS  S)cecutive  Agent  Office. 
The  Amy.  Marine  Corps,  and  Air  Force  are  requested  to  identify  a 
Senior-Xievel  individual  who  will  be  principally  responsible  within 
their  Service  for  ATS  xnanegement.  The  Director.  Weapon  Support 
Improvement  Group  (WSIG) «  OUSD(AaT).  %idll  provide  program  oversight. 

o 
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within  90  toys,  the  Navy  ahall  provide  to  the  Director,  WSIO  a 
coordinated  &cecutive  Agent  Charter  for  iscuance  as  a  DoD  Directive, 
recoctmended  organizational  and  funding  adjustments  to  implement  this 
policy,  and  proposed  acquisition  changes  to  be  incorporated  in  DoD 
Directive  9000.2.  In  addition,  the  Navy  shall  lead  a  review  of  all 
current  and  ongoing  Service  ATS  prograzns  for  application  of  the  new 
policy.  This  effort  is  to  be  conpleted  within  160  toys.  In  the 
interim,  X  ask  that  no  ATS  acquisitions  be  initiated  outside  the 
tosignated  families  or  COTS  testers  as  specified  above. 
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Audit  Report  No.  93-138,  "Quick-Reaction  Report  on  the  Acquisition  of  the 
F-15  Downsized  Tester,"  June  30,  1993.  The  report  stated  that  the  Director, 
Weapon  Support  Improvement  Gro^,  Office  of  the  Assistant  Secretary  of 
Defense  (Production  and  Logistics),  requested  the  audit.  He  r^uested  the 
audit  because  the  Air  Force  was  not  developing  a  family  of  standardized  ATE  or 
using  existing  DoD  families  of  ATE  that  met  multiple  weapon  system  and 
Military  Department  test  requirements.  The  report  stated  that  the  Air  Force's 
development  and  acquisition  of  unique  ATE  to  replace  existing  F-15  avionics 
ATE  was  not  cost-effective.  This  ineffective  cost  situation  occurred  because  the 
Air  Force  did  not  comply  with  Defense  acquisition  management  guidance  and 
perform  sufficient  cost,  technical,  and  requirements  analyses  to  determine 
whether  standard  DoD  families  of  ATE  or  other  ATE  could  satisfy  Air  Force 
ATE  requirements.  In  addition,  OSD  and  the  Air  Force  did  not  develop  and 
implement  effective  policy  and  guidance  on  standardized  ATE.  The  report 
recommended  that  the  Air  Force  discontinue  acquisition  of  unique  ATE  to 
replace  existing  F-15  avionics  ATE  and  prepare  thorough  analyses  of  costs  and 
technical  requirements  for  standardized  ATE.  The  report  also  recommended 
that  USD(A&T)  and  the  Air  Force  implement  policies  and  procedures  for 
standardized  ATE.  The  Air  Force  nonconcurred  with  the  recommendations  to 
discontinue  acquisition  of  unique  ATE  and  to  prepare  the  analyses.  The  Air 
Force  stated  that  discontinuing  the  acquisition  would  result  in  monetary  losses, 
that  an  operational  requirements  document  was  approved  by  the  Air  Force  Vice 
Chief  of  Staff,  and  that  a  Cost  and  Operational  Effectiveness  Analysis  was  not 
required.  The  Assistant  Secretary  of  Defense  (Production  and  Logistics)  and 
the  Air  Force  agreed  to  implement  policies  and  procedures  for  standardizing 
ATE.  The  USD(A&T)  directed  his  staff  to  circulate  proposed  ATS  acquisition 
policy  language  for  coordination  by  October  9,  1993.  In  the  interim,  his  staff 
would  review  ATS  acquisitions  against  the  objective  of  procuring  standardized 
in-inventory  ATE.  On  April  29,  1994,  USD(A&T)  issued  a  DoD  policy 
memorandum  for  ATS. 

Audit  Report  No.  92-095,  "Acquisition  and  Management  of  Maintenance 
and  Diagnostic  Automatic  Test  Equipment,"  May  21,  1992.  As  a  part  of  a 
DoD-wide  audit  of  the  acquisition  and  management  of  maintenance  and 
diagnostic  ATE,  the  report  evaluated  the  effectiveness  of  DoD-wide  guidance 
and  procedures  for  monitoring  the  acquisition  and  management  of  maintenance 
and  diagnostic  ATE  by  the  Militaty  Departments.  The  report  stated  that 
management  deficiencies  by  the  Military  Departments  and  the  lack  of  uniform 
and  comprehensive  DoD-wide  policy  and  guidance  contributed  to  the  continued 
proliferation  of  maintenance  and  diagnostic  ATE  and  has  seriously  effected  the 
cost-effectiveness  of  acquisitions.  The  report  recommended  that  comprehensive 
and  uniform  DoD-wide  policy  and  guidance  on  the  acquisition  and  management 
of  maintenance  and  diagnostic  ATE  be  developed  and  implemented  and  that 
OSD  oversight  responsibilities  be  established.  Management  basically  agreed  on 


*In  September  1993,  OSD  disestablished  this  office  and  created  the  Offices  of  the  Assistant 
Secretary  of  Defense  (Economic  Security),  the  Deputy  Under  Secretary  of  Defense 
(Environmental  Security),  and  the  Deputy  Under  Secretary  of  Defense  (Logistics). 
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a  need  for  uniform  and  comprehensive  DoD-wide  policy  guidelines  on  the 
acquisition  of  ATE  and  the  need  to  clarify  OSD  management  responsibilities. 
An  ATS  study  was  completed  and  a  DoD  ATS  investment  strategy  that 
capitalizes  on  existing  investments  was  developed.  The  investment  strategy  was 
documented  in  a  summary  report  that  recommended  management  changes, 
acquisition  policy  changes,  and  next-generation  technology  investments  to  verify 
implementation  and  long-term  viability  of  the  strategy.  Policy  changes  are 
being  generated  and  coordinated  at  the  USD(A&T)  level.  On  April  29,  1994, 
the  USD(A&T)  issued  a  DoD  policy  memorandum  for  ATS  (Enclosure  2). 

Audit  Report  No.  92-037,  "Effectiveness  of  the  Air  Force's  Internal 
Controls  Over  the  Development  and  Acquisition  of  Maintenance  and 
Diagnostic  Systems,"  January  23,  1992,  As  part  of  a  DoD-wide  audit  of  the 
development  and  acquisition  of  DoD  maintenance  and  diagnostic  systems,  the 
report  evaluated  the  effectiveness  of  the  Air  Force’s  principal  program  for 
monitoring  the  development  and  acquisition  of  maintenance  and  diagnostic 
systems,  the  modular  ATE  program.  The  report  stated  that  Air  Force  Systems 
Command's  product  divisions  and  Air  Force  Logistics  Command’s  logistics 
centers  were  not  complying  with  Air  Force  guidance  for  acquiring  standardized 
ATE.  As  a  result,  the  Air  Force  experienced  a  continued  proliferation  of 
equipment  and  provided  no  assurance  of  acquiring  ATE  cost-effectively.  The 
report  recommended  that  the  Air  Force  develop  and  implement  an  effective 
internal  control  management  system  for  monitoring  the  development  and 
acquisition  of  ATE.  The  Air  Force  concurred  with  the  report  and  accordingly 
issued  Air  Force  Policy  Directive  63-2,  "Automatic  Test  Systems  and 
Equipment,"  July  19,  1994.  This  Directive  established  responsibilities  and 
authorities  for  developing  and  implementing  an  effective  ATS  development  and 
acquisition  internal  control  management  system  as  recommended  in  the  report. 

Audit  Report  No.  92-022,  "Development  and  Acquisition  of  DoD 
Maintenance  and  Diagnostic  Systems-Navy,"  December  17,  1991.  The 
report  stated  that  the  Navy’s  plans  for  transitional  to  standard  automatic  test 
equipment  (ATE)  developed  under  its  Consolidated  Automated  Support  System 
Program  have  not  been  fully  effective.  As  a  result,  potential  savings 
opportunities  have  been  missed  because  workload  and  economic  analyses  were 
not  performed  by  several  Navy  activities  to  determine  whether  it  was  feasible 
and  economical  to  transition  from  existing  test  equipment  for  their  weapon 
systems  to  Consolidated  Automated  Support  System  test  equipment.  The  report 
recommended  that  the  Navy  develop  an  effective  internal  control  management 
system  to  monitor  the  Navy-wide  development,  acquisition,  and  distribution  of 
test,  measurement,  and  diagnostic  equipment.  The  Navy  nonconcurred  with  the 
finding  and  all  recommendations.  In  mediation,  the  Navy  stated  that: 

o  it  was  constructing  a  database  of  all  ATE; 

o  the  Office  of  the  Assistant  Secretary  of  the  Navy  (Research, 
Development  and  Acquisition)  was  assigned  management  responsibility  for  all 
Navy  ATE; 

o  managers  were  required  to  justify  the  use  of  non-CASS  equipment; 
and 
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o  managers  will  reuse  CASS  equipment  after  test  program  set 
development. 

Also,  the  Navy  agreed  to  use  a  modified  Naval  Air  Systems  Command  strategy 
for  the  transition  to  ATE. 
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Organizations  Visited  or  Contacted 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology,  Washington,  DC 
Principal  Deputy  Under  Secretary  of  Defense  (Acquisition  and  Technology), 
Washington,  DC 

Director,  Acquisition  Program  Integration,  Washington,  DC 
Director,  Defense  Research  and  Engineering,  Washington,  DC 
Assistant  Secretary  of  Defense  (Economic  Security),  Washington,  DC 
Director,  Weapon  Support  Improvement  Group,  Washington,  DC 
Deputy  Under  Secretary  of  Defense  (Logistics),  Washington,  DC 
Director,  Operational  Test  and  Evaluation,  Washington,  DC 


Department  of  the  Army 

Army  Materiel  Command,  Alexandria,  VA 

Director,  U.S.  Army  Test  Measurement  and  Diagnostic  Equipment  Activity, 
Redstone  Arsenal,  AL 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Research,  Development  and  Acquisition), 
Washington,  DC 

Naval  Air  Systems  Command,  Arlington,  VA 

Aviation  Support  Equipment  Office,  Arlington,  VA 
Naval  Air  Warfare  Center,  Aircraft  Division,  Lakehurst,  NJ 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Acquisition),  Washington,  DC 


Defense  Agency 

Defense  Logistics  Agency,  Alexandria,  VA 


ENCLOSURE  4 


Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 

Principal  Deputy  Under  Secretary  of  Defense  (Acquisition  and  Technology) 
Director,  Acquisition  Program  Integration 
Director,  Defense  Research  and  Engineering 
Assistant  Secretary  of  Defense  (Economic  Security) 

Director,  Weapon  Support  Improvement  Group 
Deputy  Under  Secretary  of  Defense  (Acquisition  Reform) 

Deputy  Under  Secretary  of  Defense  (Logistics) 

Under  Secretary  of  Defense  (Comptroller) 

Director,  Operational  Test  and  Evaluation 
Assistant  to  the  Secretary  of  Defense  (Public  Affairs) 


Department  of  the  Army 

Secretary  of  the  Army 

Assistant  Secretary  of  the  Army  (Research,  Development  and  Acquisition) 
Army  Materiel  Command 

Director,  U.S.  Army  Test  Measurement  and  Diagnostic  Equipment  Activity 
Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Secretary  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Research,  Development  and  Acquisition) 
Comptroller  of  the  Navy 
Naval  Air  Systems  Command 

Aviation  Support  Equipment  Office 
Naval  Air  Warfare  Center,  Aircraft  Division 
Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Secretary  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Acquisition) 

Auditor  General,  Department  of  the  Air  Force 
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Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 

Director,  Defense  Logistics  Agency 

Director,  National  Security  Agency 

Inspector  General,  Central  Imagery  Office 

Inspector  General,  Defense  Intelligence  Agency 

Inspector  General,  National  Security  Agency 

Director,  Defense  Logistics  Studies  Information  Exchange 


Non-Defense  Federal  Organizations  and  Individuals 

Office  of  Management  and  Budget 

Technical  Information  Center,  National  Security  and  International  Affairs  Division, 
General  Accounting  Office 

Chairman  and  Ranking  Minority  Member  of  Each  of  the  Following  Congressional 
Committees  and  Subcommittees: 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Operations 

House  Subcommittee  on  Legislation  and  National  Security,  Committee  on 
Government  Operations 
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Audit  Team  Members 


Donald  E.  Reed 
Russell  A.  Rau 
John  E.  Meling 
Jack  D.  Snider 
Alvin  B.  Lowe 
Cordelia  Grace-Scott 
Debbie  A.  Calhoun 
Scott  A.  Marx 
Mary  Ann  Hourcl6 
Teresa  D.  Bone 
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